Video decoding apparatus and method

ABSTRACT

A video decoding apparatus for decoding an encoded video bitstream having frames of video data encoded in rows of macroblocks. The video decoding apparatus comprises a parsing unit configured to receive the encoded video bitstream and to interpret the encoded video bitstream to generate items of macroblock information to be used for reconstructing the video frames of video data. The parsing unit is configured to store the items of macroblock information in a memory in bitstream order. The video decoding apparatus further comprises a line control unit configured to generate line control information associated with each row of macroblocks, the line control information comprising a sequence of pointers to the items of macroblock information stored in the memory, such that sequentially reading the sequence of pointers accesses the items of macroblock information in raster scan order. The line control information is stored in said memory in association with said items of macroblock information. A reconstruction pipeline is configured to reconstruct the frames of video data with reference to the line control information.

This application claims priority to UK Application No. 1013625.7 filed13 Aug. 2010, the entire content of which is hereby incorporated byreference.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to decoding an encoded video bitstream, inparticular where the encoded video bitstream represents frames of videodata encoded in rows of macroblocks.

Description of the Prior Art

Contemporary video encoding techniques, such as those represented by theH.264 “Advanced Video Coding” standard, provide for highly efficientencoding of video data. As such, the decoding process for decoding anencoded video bitstream will typically comprise several stages, frominitially interpreting the bit pattern of the bitstream, throughextracting information related to individual macroblocks, toreconstructing an entire frame of video data on the basis of thosemacroblocks.

In some known video decoders, the video decoding process is split intotwo phases, a first parsing phase in which the received encoded videobitstream is initially interpreted in order to generate macroblockinformation, and a second pipelined stage in which the macroblockinformation is processed and combined to reconstruct individual framesof video data.

Whilst it may be the case that the macroblocks in an encoded videobitstream are represented in that bitstream in the same order as thosemacroblocks appear in the frames of video data (i.e. in raster scanorder), video encoding standards such as H.264 permit a FlexibleMacroblock Order (FMO), wherein the order in which the macroblocks areencoded in the encoded video bitstream does not correspond to rasterscan order. This may for example occur when a frame of video data isencoded in more than one slice, wherein those slices overlap oneanother. For example, a checkerboard pattern of slices is possible,wherein alternate macroblocks belong to two separate slices of encodedvideo data. In other words, a video decoding apparatus receiving such anencoded video bit stream will first receive information related to odd(say) numbered macroblocks for the frame, followed by informationrelated to the even numbered macroblocks of that frame. Suchinterleaving of macroblock slices has advantages in terms of errorresilience, because even if a slice of video data is lost intransmission, a reasonable approximation to the original frame of videodata may nonetheless be reconstructed by interpolating to generate themissing data. For this reason, these techniques find application inenvironments where the transmission medium is known to be lossy, butwhere the absolute quality of the reconstructed video data is of lesserimportance, such as in mobile video conferencing.

However, allowing FMO in the encoded video bitstream presents the videodecoding apparatus with an increased level of complexity. In particular,a problem arises when the parsing of the encoded video bitstream occursmost efficiently when it is carried out in bitstream order. This can forexample be because each slice of encoded video may make reference toitself in its encoding, for example a given macroblock row in one slicemay refer to the previous row in that slice. Hence, for the parser, itis most efficient if the identified items of macroblock information arewritten to memory in bitstream order, such that as the parsing processcontinues, the parser may easily make reference to earlier identifiedmacroblocks of the same slice. Furthermore, handling the slice as it isreceived enables the parser to maintain a consistent context for itsentropy decoder, thus avoiding the extra bandwidth and processingassociated with context switching between slices.

On the other hand, the reconstruction of the identified macroblocks intothe frames of video data taking place in the reconstruction pipelinehappens most efficiently when it is carried out in raster scan order.This is because it is the raster scan order that defines where themacroblocks lie in relation to one another in the final frames of videodata and sequential access to these macroblocks typically permits themost efficient use of, for example, the frame buffer and the motioncache.

FIG. 1 schematically illustrates a known video decoding apparatus 100 inwhich a video engine 110 comprising a parser 120 and a reconstructionpipeline 130 receives an encoded video bitstream and decodes it,typically for display. The video engine 110 is coupled to memory 140which the parser 120 and reconstruction pipeline 130 make use of inperforming their parts in the decoding operation. As can be seen in FIG.1, both the parser 120 and reconstruction pipeline 130 are arranged withfeedback paths, wherein some of the information they output is fed backas an input (NB these feedback paths are merely schematic, the use ofpreviously output information as an input typically taking place bymeans of an access to memory 140). It is in particular these feedbackloops which determine that parser 120 operates most efficiently in abitstream order and reconstruction pipeline 130 operates mostefficiently in raster scan order.

One approach to unify this situation is to configure the parser to parsethe encoded video bitstream in raster scan order. Although thissimplifies the overall control of the video encoder, it has thedisadvantage that the parsing cannot start until the bitstream for theentire frame has been received, which increases the latency of the videodecoder. Furthermore when interleaved slices are received, the processof switching between different slices on a macroblock by macroblockbasis involves the above mentioned context switching in the entropydecoder and consequently increases memory access bandwidth.

Alternatively, it would be possible to allow the parser to operate inbitstream order, and also to perform some of the reconstruction inbitstream order, namely decoding slices into pixels, followed by runningthe deblocking as a second pass. However, this has the disadvantage thatthe access to the frame buffer (which is the highest bandwidth access inthe process) is no longer sequential and therefore inefficient.Furthermore the motion cache in the reconstruction pipeline is alsopoorly utilised.

Some background information on the technological issues involved can befound in the Wikipedia article “Arbitrary slice ordering” (retrievedfrom http://en.wikipedia.org/wiki/Arbitrary_slice_ordering on 26 Mar.2010). The paper “Macroblock-level decoding and deblocking method andits pipeline implementation in H.264 decoder SOC design” Wang S. et al.(Journal—Zhejiang University Science A 2007, Vol. 8, Number 1, pages36-41) is concerned with the problems raised by FMO and presents amulti-stage video decoder which allows the later phases to run in rasterscan order. However, the approach taken involves searching the inputbitstream which can be very costly in the case of an entropy encodedbitstream (e.g. CABAC), and involves switching between slices which isexpensive in terms of memory accesses.

Accordingly, it would be desirable to provide a technique which enabledeach stage of a video decoder to operate in an efficient configuration,avoiding the above described disadvantages of handling data in an orderpreferred by another part of the video decoder.

SUMMARY OF THE INVENTION

Viewed from a first aspect, the present invention provides a videodecoding apparatus for decoding an encoded video bitstream, said encodedvideo bitstream representing frames of video data encoded in rows ofmacroblocks, said video decoding apparatus comprising: a parsing unitconfigured to receive said encoded video bitstream, said parsing unitcomprising an interpretation unit configured to interpret said encodedvideo bitstream to generate items of macroblock information to be usedfor reconstructing said frames of video data, said parsing unitconfigured to store said items of macroblock information in a memory inbitstream order; a line control unit configured to generate line controlinformation associated with each row of macroblocks, said line controlinformation comprising a sequence of pointers to said items ofmacroblock information stored in said memory, such that sequentiallyreading said sequence of pointers accesses said items of macroblockinformation in raster scan order, said line control unit configured tostore said line control information in said memory in association withsaid items of macroblock information; and a reconstruction pipelineconfigured to reconstruct said frames of video data with reference tosaid line control information.

According to the techniques of the present invention the parsing unit ofthe video decoding apparatus receives the encoded video bitstream whichcontains information relating to the rows of macroblocks in which theframes of video data are encoded. The parsing unit, by means of itsinterpretation unit, interprets the encoded video bitstream andgenerates items of macroblock information which are written into amemory in the same order in which they are received in the encoded videobitstream. Hence, the parsing unit operates on macroblocks in bitstreamorder, handling each macroblock as it arrives, without the need toperform any reordering of the macroblocks.

A line control unit is provided which generates line control informationassociated with each row of macroblocks. The line control unit generatesa sequence of pointers to items of macroblock information stored in thememory, these pointers being arranged in the sequence such thatsequentially reading the sequence of pointers accesses items ofmacroblock information in raster scan order. This line controlinformation is stored in the memory in association with the items ofmacroblock information, meaning that when those items of macroblockinformation are accessed, the line control information may also beaccessed, in particular allowing access to the items of macroblockinformation to take place in raster scan order. This is particularlybeneficial in the reconstruction pipeline of the video decodingapparatus, which operates most efficiently when reconstructing frames ofvideo data on the basis of macroblock information when doing so inraster scan order of the macroblocks.

Hence, the parsing unit is able to handle the macroblock information inbitstream order, efficiently storing the items of macroblock informationin the memory in the same order in which they are received in theencoded video bitstream, without the need to perform any reordering. Atthe same time, the provision of the line control information inassociation with the items of macroblock information stored in thememory means that subsequent access to those items of macroblockinformation, such as by the reconstruction pipeline when reconstructingframes of video data, may take place in raster scan order, thus allowingthe reconstruction to operate in an efficient configuration, whichreduces the energy consumption of the video decoding apparatus andreduces the memory bandwidth required within the video decodingapparatus.

The techniques of the present invention are of particular benefit whenthe order in which the macroblocks are encoded in the encoded videobitstream does not correspond to the raster scan order in which thosemacroblocks appear in the frames of video data since the macroblocks mayon the one hand be read directly from the memory in bitstream order(having been written into the memory in bitstream order), but may on theother hand also be retrieved from the memory (by reference to the linecontrol information) in raster scan order. Whilst the techniques of thepresent invention are also applicable in situations where the bitstreamorder is the same as the raster scan order (and in which the linecontrol information then simply indicates a sequence of items ofmacroblock information which are already in raster scan order in theorder in which they have been stored in memory), in embodiments of thepresent invention the bitstream order and the raster scan order aredifferent from one another.

The interpretation of the encoded video bitstream carried out by theinterpretation unit of the parsing unit may take a number of forms, inparticular in dependence on the particular codec which has been used toencode the video bitstream, but in embodiments of the present inventionthe interpretation unit comprises a prediction resolver configured toresolve predictive encoding between macroblocks of said encoded videostream. Predictive encoding between macroblocks of the encoded videostream is one technique which allows the video bitstream to be moreefficiently encoded, since the information content of one macroblock maybe used to predict the information content of another macroblock andthus reduce the amount of information required to encode the lattermacroblock. Such predictive encoding may, for example, comprise rowprediction, in which the macroblocks of a current row of macroblockshave been encoded with respect to a previous row of macroblocks withinthe same frame, or may comprise temporal (or “co-located”) prediction inwhich a macroblock in one frame is predicted with respect to amacroblock in the same position in a previous frame.

In some embodiments, said parsing unit is further configured whenstoring an item of macroblock information in said memory, if said itemof macroblock information does not sequentially follow in raster scanorder a preceding item of macroblock information in said memory, tostore a marker in said memory preceding said item of macroblockinformation, said marker indicative of a number of macroblocks whichintercede said preceding item of macroblock information and said item ofmacroblock information in raster scan order.

Accordingly, when the parsing unit stores an item of macroblockinformation in the memory, if that item of macroblock information doesnot sequentially follow in raster scan order the preceding item ofmacroblock information stored in the memory, for example because theslice of video data currently being decoded encodes alternatemacroblocks in a frame in a checkerboard fashion, the parsing unit isconfigured to store a marker preceding the item of macroblockinformation currently being decoded, wherein the marker indicates howmany macroblocks have been skipped when viewed in raster scan order.Hence, whilst on the one hand the parsing unit continues to store theitems of macroblock information in the memory in the order in which theyare received in the encoded video bitstream, i.e. in bitstream order,the additional markers stored in the memory indicate the layout of theitems of macroblock information when viewed in raster scan order, i.e.in the original frame of video data. This additional information storedwith the items of macroblock information allows them to be readsequentially in the memory, but including the knowledge of how theyappear in the frame of video data.

Storing a marker in the memory in this fashion can have a number ofadvantages, and in one such embodiment said prediction resolver isconfigured to resolve said predictive encoding between macroblocks ofsaid encoded video stream with reference to a sequence of items ofmacroblock information stored in said memory including at least one saidmarker.

The presence of the at least one marker in the sequence of items ofmacroblock information which the prediction resolver refers to enablesthe prediction resolver to resolve the predicted encoding by knowingwhere the macroblocks lie in the frame of video data, but whilst stillreading them in sequential order. For example, where the predictionresolver needs to refer to the previous row of macroblocks, the markerenables the prediction resolver to determine the relative location ofthe read macroblocks from that previous row.

It will be appreciated that the marker could take a number of forms, butin one embodiment the marker has an item of macroblock information dataformat. Using the same data format for the marker as for the item ofmacroblock information has advantages for both writing and reading theitems of macroblock information, since the same data format will bepresent whether a marker or a genuine item of macroblock information isbeing written/read.

When the marker has an item of macroblock information data format, insome embodiments the marker includes a flag indicating that said markeris not an item of macroblock information. This enables a simplifiedreading process wherein items of macroblock information and markers maybe easily distinguished.

It will be recognised that the flag could take a number of forms, but inone embodiment the flag comprises a zero length indication. Items ofmacroblock information may have differing lengths depending on thecomplexity of the data encoded therein, but a genuine item of macroblockinformation will have finite length, and hence using a zero lengthindication provides a convenient mechanism for implementing the flagshowing that the marker is not a true item of macroblock information.

The provision of the line control information enables subsequentprocesses to access the items of macroblock information in raster scanorder, such as those carried out in the reconstruction pipeline, andhence any markers stored in the memory (effectively indicating adifference between bitstream order and raster scan order of themacroblocks) are not required by those subsequent processes. Whilst itwould be possible to configure the reconstruction pipeline to ignoresuch markers, it is advantageous if they are simply not seen in thereconstruction pipeline. Hence in some embodiments said line controlunit is configured to generated said sequence of pointers such that saidreconstruction pipeline does not encounter said marker whenreconstructing said frames of video data with reference to said linecontrol information. In other words following the sequence of pointersdoes not result in a marker being read.

It will be appreciated that the particular form that the marker takeswhen stored in the memory may take a variety of forms, but in someembodiments the marker is run length compressed. The marker has arelatively simple task to perform and hence it is advantageous if itonly occupies a small amount of memory, and run length compression is anefficient way of achieving this.

It will be appreciated that the line control information can take anumber of forms, but in embodiments of the present invention the linecontrol information comprises a vector, said vector having a lengthgiven by how many rows of macroblocks are in each frame of video data,each element of said vector indicating said sequence of pointers for acorresponding row of that frame.

Accordingly, one vector can be provided for each frame of video data,the length of the vector being determined by the number of rows ofmacroblocks in the frame, and each element of the vector then points tothe sequence of pointers corresponding to each row. This presents aformat in which the line control information may be easily managed,written and read.

In some such embodiments, a first pointer indicated by said vector isstored in a first element of said vector. Whilst the vector may indicatea number of pointers for a given row of the frame, it may be the casethat only one pointer is required for a given row, when for that row theitems of macroblock information stored in the memory are already inraster scan order (i.e. the bitstream order corresponded to raster scanorder). In this situation the reading process for the line controlinformation is simplified if the single pointer required for this row isstored in the vector itself.

In some embodiments said line control unit is further configured toinclude in said line control information an indication of how manypointers are comprised in said sequence of pointers. Providing anindication of how many pointers are in the sequence facilitates thereading process for that sequence of pointers.

In some embodiments said line control unit is further configured toinclude in each pointer of said sequence of pointers an indication ofhow many sequential macroblocks are pointed to by that pointer. Anindication of how many sequential macroblocks are pointed to by eachpointer enables a later process reading the items of macroblockinformation from the memory by means of the line control information tosimply read the items of macroblock information, without the need forthat later process to identify the end of each run of sequentialmacroblocks, for example by reading until a marker is encountered.

The sequence of pointers indicating the raster scan order of the itemsof macroblock in memory further provides a mechanism by which errors inthe bitstream can be efficiently handled, and in some embodiments saidline control unit is further configured, if said parsing unit determinesthat said encoded video bitstream contains an error corresponding to atleast one item of macroblock information, to omit a correspondingpointer to said at least one item of macroblock information from saidsequence of pointers. Hence, items of macroblock information that havebeen corrupted as a result of errors in the encoded video bitstream cansimply and efficiently be omitted from a later reading process by meansof a corresponding pointer being omitted from the sequence of pointers.

The prediction resolver in the interpretation unit could take a numberof forms, but in some embodiments the prediction resolver is configuredto resolve temporal predictive encoding with respect to a previouslydecoded frame of video data, and said prediction resolver is configuredto retrieve items of macroblock information corresponding to saidpreviously decoded frame of video data by reference to said line controlinformation. Hence, the line control information provides an efficientmechanism for the prediction resolver to access a previously decodedframe of video data that has already been stored in the memory.

It will be recognised that the prediction resolver could be configuredto perform various different types of prediction, and in someembodiments said prediction resolver is configured to perform motionvector prediction. Further, in some embodiments the prediction resolveris configured to perform intra-mode prediction.

The line control information provides for a particularly efficientreading mechanism for the items of macroblock information stored in thememory, and in some embodiments said reconstruction pipeline isconfigured to perform parallel decoding on said frames of video data andto reference said line control information to read items of macroblockinformation in parallel from said memory. Parallel decoding performed bythe reconstruction pipeline allows a greater throughput of video datathrough the video decoding apparatus.

Such parallel reading of macroblock information from the memory couldtake place in a number of ways. In one embodiment said reconstructionpipeline is configured to reference said line control information toread said frames of video data in parallel from said memory. In anotherembodiment said reconstruction pipeline is configured to reference saidline control information to read said rows of macroblock information inparallel from said memory.

Viewed from a second aspect the present invention provides a method ofdecoding an encoded video bitstream, said encoded video bitstreamrepresenting frames of video data encoded in rows of macroblocks, themethod comprising the steps of: receiving said encoded video bitstream;interpreting said encoded video bitstream to generate items ofmacroblock information to be used for reconstructing said frames ofvideo data; storing said items of macroblock information in a memory inthe same order in which they are received in said encoded videobitstream; generating line control information associated with each rowof macroblocks, said line control information comprising a sequence ofpointers to said items of macroblock information stored in said memory,such that sequentially reading said sequence of pointers accesses saiditems of macroblock information in raster scan order; storing said linecontrol information in said memory in association with said items ofmacroblock information; and reconstructing said frames of video datawith reference to said line control information.

Viewed from a third aspect, the present invention provides anon-transitory computer readable storage medium storing a computerprogram, which when loaded onto a computing device causes said computingdevice to carry out the method of the second aspect.

Viewed from a fourth aspect the present invention provides a videodecoding apparatus for decoding an encoded video bitstream, said encodedvideo bitstream representing frames of video data encoded in rows ofmacroblocks, said video decoding apparatus comprising: a parsing meansfor receiving said encoded video bitstream, said parsing meanscomprising an interpretation means for interpreting said encoded videobitstream to generate items of macroblock information to be used forreconstructing said frames of video data, said parsing means for storingsaid items of macroblock information in a memory in the same order inwhich they are received in said encoded video bitstream; a line controlmeans for generating line control information associated with each rowof macroblocks, said line control information comprising a sequence ofpointers to said items of macroblock information stored in said memory,such that sequentially reading said sequence of pointers accesses saiditems of macroblock information in raster scan order, said line controlmeans for storing said line control information in said memory inassociation with said items of macroblock information; and areconstruction pipeline means for reconstructing said frames of videodata with reference to said line control information.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be described further, by way of example only,with reference to embodiments thereof as illustrated in the accompanyingdrawings, in which:

FIG. 1 schematically illustrates a video decoding apparatus according tothe prior art;

FIG. 2 schematically illustrates some components of a parsing unit and amemory in a video decoding apparatus in one embodiment;

FIG. 3 schematically illustrates macroblock information in raster order,in bitstream order, and how items of macroblock information are storedin memory in one embodiment;

FIG. 4 schematically illustrates a checkerboard pattern of a slice ofencoded video data and its corresponding storage pattern in the mbinfobuffer in one embodiment;

FIG. 5 shows an example content of the mbinfo buffer in memory withoutflexible macroblock ordering in one embodiment;

FIG. 6 illustrates an example content of the mbinfo buffer in memorywith flexible macroblock ordering in one embodiment; and

FIG. 7 schematically illustrates a series of steps taken by a parsingunit in one embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG. 2 schematically illustrates various components of a video decodingapparatus 200 according to one embodiment. For simplicity, only theparser 205 and a memory 210 are illustrated. The other main significantcomponent of the video decoding apparatus, namely the reconstructionpipeline, has been omitted for clarity.

The parser 205 receives an encoded video bitstream via buffer 215 whichtemporarily stores the encoded video bitstream before it is passed tothe substantive parts of the parser 205 which perform the interpretationof the encoded video bitstream. These further components areschematically represented by bit decoder 220, interpreter 225 andprediction unit 230. Together bit decoder 220, interpreter 225 andprediction unit 230 can generally be considered to form aninterpretation unit.

Bit decoder 220 is configured to perform an initial interpretation ofthe sequence of bits which forms the bitstream, for example translatingreceived packed sequences of bits into corresponding unpacked versions.The interpreter unit 225 begins the true interpretation process byidentifying macroblock information within the bitstream and extractingresidual information. The residual information is written by interpreter225 into the residuals buffer 240 in memory 210. The final stage ofinterpretation within the parser 205 is carried out by prediction unit230. Prediction unit 230 generates the items of macroblock information(mbinfo) which it then writes into the mbinfo buffer 250 in memory 210.Part of the prediction carried out by prediction unit 230 comprisestemporal prediction, wherein co-located macroblock information from apreviously decoded frame is retrieved from mbinfo buffer 250. Anotherkind of prediction performed by prediction unit 230 is row prediction,wherein items of macroblock information belonging to a previous row ofthe frame currently being decoded are retrieved and used to predict thecontent of the current row being decoded.

Parser 205 also comprises line control unit 235. Line control unit 235monitors the items of macroblock information generated by predictionunit 230 and intermittently generates line control information which itstores in mbinfo buffer 250 in association with the items of macroblockinformation being written there by prediction unit 230. Importantly, theline control unit 235 generates the line control information comprisinga sequence of pointers to the items of macroblock information stored inmbinfo buffer 250 by prediction unit 230, the sequence of pointers beingsuch that when these are read in order the items of macroblockinformation are accessed in raster scan order. Hence, for example, whenthe reconstruction pipeline accesses mbinfo buffer 250 in memory 210 itcan do this by means of the line control information (mbinfo_line) tothen retrieve items of macroblock information in raster scan order.Conversely note that parsing unit 205 writes the items of macroblockinformation into memory 210 in the order in which it receives them inthe bitstream.

The retrieval of items of macroblock information from memory 210 by thereconstruction pipeline can also be parallelised (schematicallyillustrated by the dashed access line in FIG. 2). Thus, more than oneitem of line control information (mbinfo_line) can be accessed by thereconstruction pipeline at a time to increase the reconstructionthroughput. This may involve accessing multiple lines of macroblockinformation in parallel, and/or accessing multiple frames in parallel.

Parsing unit 205 is further configured to store a marker between itemsof macroblock information stored in mbinfo buffer 250 when those twoitems of macroblock information do not sequentially follow one anotherin raster scan order. These markers are discussed in more detail withreference to FIG. 3.

A difference between raster scan order and bitstream order for asequence of macroblocks is schematically illustrated in FIG. 3. The setof six macroblocks 300 is encoded in two slices (illustrated by thehatched and dotted boxes respectively). This then means that inbitstream order 305 the hatched macroblocks 1, 3 and 6 are receivedfirst, followed by the dotted macroblocks 2, 4 and 5. The sequence ofmacroblocks 310 then illustrates how these items of macroblockinformation are written into mbinfo buffer 250. Reading from left toright, macroblock 1 is written first, followed by a marker indicatingthat the next item of macroblock information does not immediately followin raster scan order. This marker has the same data format as an item ofmacroblock information but in its header is labelled as having a zerolength (len=0) and indicates how many macroblocks have been skipped(when viewed in raster scan order). Similarly, macroblock 3 is followedby a marker indicating that two macroblocks have been skipped. Writingthe items of macroblock data into the mbinfo buffer in this mannerallows the parsing unit to handle the macroblocks in the bitstream order305 in which they are received. Furthermore, when prediction unit 230needs to read these items of macroblock data in order to carry out aprediction operation, the items of macroblock information 310 can simplybe read in order.

Line control unit 235 monitors the items of macroblock information beingwritten into mbinfo buffer 250. In particular, when a complete row ofmacroblocks has been written into mbinfo buffer, the line control unit235 writes the line control information mbinfo_line into the mbinfobuffer 250 in association with the items of macroblock informationcorresponding to that line. The line control information mbinfo_linecomprises a vector indicating a sequence of pointers (mbinfo_strip)which each point to an item of macroblock information in the mbinfobuffer 250. The mbinfo_strip pointers are arranged such that when thissequence of pointers 320 is read, the items of macroblock informationreferred to are read in raster scan order.

FIG. 4 schematically illustrates a frame of macroblock information 400in which alternate macroblocks are encoded in a first slice in acheckerboard fashion. A slice encoding like this may be of particularbenefit when used in the context of a lossy transmission medium, so thatdata loss in transmission can be accommodated and a reasonable qualityof decoded video (e.g. by interpolation) can still be provided. Asdescribed above the items of macroblock information 1, 3 and 5 arewritten into the mbinfo buffer 250 including intervening markersindicating that a macroblock has been skipped between macroblock 1 andmacroblock 3, and between macroblock 3 and macroblock 5. A markerfollows macroblock 5 indicating that macroblock 6 is skipped in thisslice. The series of items of macroblock information 410 illustrated inFIG. 4 shows how a previously parsed row of macroblock information (1,3, 5) is read whilst the next row of macroblock information (8, 10, 12)is being parsed and written to the mbinfo buffer. This is part of therow prediction process, for example where macroblock 8 is interpretedwith reference to adjacent macroblocks 1 and 3 in the previous row. Notealso that between macroblocks 5 and 8 there are two “skip one”indicators (instead of one “skip two” indicator) since this enables eachrow to be handled individually.

FIG. 5 illustrates an example content of the mbinfo buffer stored in thembinfo buffer in a situation where the encoded video bitstream receiveddoes not include flexible macroblock ordering. The content begins with aframe header which contains basic frame information (e.g. frame type,interlace method etc.) and indicates where other information for theframe can be found. Following the frame header in the mbinfo structureis the mbinfo_line vector. This vector has a length corresponding to thenumber of macroblock lines in the frame and contains information on howto find the strips for a line. This information is in a form of apointer indicating where the corresponding items of macroblockinformation are stored. The mbinfo_line vector is followed by a sliceheader which provides information (e.g. slice ID, number of macroblocksand slice type) for the following slice. The items of macroblockinformation belonging to that slice then follow. The entire frame isbuilt up in this fashion, with a slice header proceeding a set of itemsof macroblock information belonging to that slice. In addition, where aslice change occurs within a line an mbinfo_strip item is used.

FIG. 6 illustrates the content of the mbinfo buffer for a frame ofmacroblock information which does include flexible macroblock ordering.The general structure is similar to that described with reference toFIG. 5, with the exception of the sequence of mbinfo_strip items storedin sequence at the end. These mbinfo_strip items are generated by theline control unit once a full line has been completed and thembinfo_line vector can be populated with pointers to each of thesembinfo_strip items (as described with reference to FIG. 3).

FIG. 7 schematically illustrates a series of steps taken by the parser205 (illustrated in FIG. 2). The flow begins at step 700 where theparser receives an encoded video bitstream, which is temporarilybuffered in buffer 215. At step 705 the first stages of interpretation(i.e. bit decoding and residual extraction) are performed. Then at step710 it is determined if there is prediction required (by prediction unit230) in order to generate the items of macroblock information from thereceived encoded video bitstream. If prediction is required the flowproceeds via step 715 where temporal and/or row prediction is performedwith reference to previously stored items of macroblock informationretrieved from mbinfo buffer 250 stored in memory 210. The flow thenproceeds to step 720 (where the flow also directly proceeds to from step710 if no prediction is required). At step 720 items of macroblockinformation (mbinfo) are generated and at step 725 are passed to mbinfobuffer 250 for storage. They are also passed to line control unit 235,which monitors the line-by-line build up of macroblocks. At step 730 itis determined by line control unit 235 if a complete row of macroblockinformation has now been stored in the mbinfo buffer 250. If it has notthen the flow simply returns to step 700. When a complete row ofmacroblock information has been stored in the mbinfo buffer 250, at step735 the line control unit 235 generates the mbinfo_line vector and therequired mbinfo_strip pointers and stores these in the mbinfo buffer250. It should be noted that generating the mbinfo_line vector willtypically comprise populating the elements of the vector which has beenpreviously created.

Hence, according to the techniques of the present invention, in a videodecoding apparatus the parsing unit is able to handle the macroblockinformation in bitstream order, whilst the reconstruction pipeline canhandle the macroblock information in raster scan order. As such, bothcomponents are able to operate in their more efficient configuration,reducing the energy consumption of the video decoding apparatus andreduces the memory bandwidth required within the video decodingapparatus.

Although a particular embodiment has been described herein, it will beappreciated that the invention is not limited thereto and that manymodifications and additions thereto may be made within the scope of theinvention. For example, various combinations of the features of thefollowing dependent claims could be made with the features of theindependent claims without departing from the scope of the presentinvention.

We claim:
 1. A video decoding apparatus for decoding an encoded videobitstream, said encoded video bitstream representing frames of videodata encoded in rows of macroblocks, said video decoding apparatuscomprising: a parser configured to receive said encoded video bitstream,interpret said encoded video bitstream to perform entropy decoding,generate items of macroblock information to be used for reconstructingsaid frames of video data, and store said items of macroblockinformation in a memory in bitstream order; a line control processorconfigured to generate line control information associated with each rowof macroblocks, said line control information configured such thatsequentially reading said line control information provides access tosaid items of macroblock information in raster scan order, said linecontrol processor configured to store said line control information insaid memory in association with said items of macroblock information,wherein said parser is configured to resolve predictive encoding betweenmacroblocks of said encoded video stream while reading said macroblocksin sequential order; and a reconstruction pipeline configured toreconstruct said frames of video data with reference to said linecontrol information.
 2. The video decoding apparatus as claimed in claim1, wherein said bitstream order and said raster scan order differ fromone another.
 3. The video decoding apparatus as claimed in claim 1,wherein said parser is further configured, when storing an item ofmacroblock information in said memory, if said item of macroblockinformation does not sequentially follow in raster scan order apreceding item of macroblock information in said memory, to store amarker in said memory preceding said item of macroblock information,said marker indicative of a number of macroblocks which intercede saidpreceding item of macroblock information and said item of macroblockinformation in raster scan order.
 4. The video decoding apparatus asclaimed in claim 1, wherein said parser is configured to resolve saidpredictive encoding between macroblocks of said encoded video streamwith reference to a sequence of items of macroblock information storedin said memory including at least one said marker.
 5. The video decodingapparatus as claimed in claim 3, wherein said marker has an item ofmacroblock information data format.
 6. The video decoding apparatus asclaimed in claim 5, wherein said marker includes a flag indicating thatsaid marker is not an item of macroblock information.
 7. The videodecoding apparatus as claimed in claim 6, wherein said flag comprises azero length indication.
 8. The video decoding apparatus as claimed inclaim 3, wherein said line control processor is configured to generatesaid sequence of pointers such that said reconstruction pipeline doesnot encounter said marker when reconstructing said frames of video datawith reference to said line control information.
 9. The video decodingapparatus as claimed in claim 3, wherein said marker is run lengthcompressed.
 10. The video decoding apparatus as claimed in claim 1,wherein said line control information comprises a vector, said vectorhaving a length given by how many rows of macroblocks are in each frameof video data, each element of said vector indicating said sequence ofpointers for a corresponding row of that frame.
 11. The video decodingapparatus as claimed in claim 10, wherein a first pointer indicated bysaid vector is stored in a first element of said vector.
 12. The videodecoding apparatus as claimed in claim 1, wherein said line controlprocessor is further configured to include in said line controlinformation an indication of how many pointers are comprised in saidsequence of pointers.
 13. The video decoding apparatus as claimed inclaim 1, wherein said line control processor is further configured toinclude in each pointer of said sequence of pointers an indication ofhow many sequential macroblocks are pointed to by that pointer.
 14. Thevideo decoding apparatus as claimed in claim 1, wherein said linecontrol processor is further configured, if said parser determines thatsaid encoded video bitstream contains an error corresponding to at leastone item of macroblock information, to omit a corresponding pointer tosaid at least one item of macroblock information from said sequence ofpointers.
 15. The video decoding apparatus as claimed in claim 1,wherein said parser is configured to resolve temporal predictiveencoding with respect to a previously decoded frame of video data, andto retrieve items of macroblock information corresponding to saidpreviously decoded frame of video data by reference to said line controlinformation.
 16. The video decoding apparatus as claimed in claim 1,wherein said parser is configured to perform motion vector prediction.17. The video decoding apparatus as claimed in claim 1, wherein saidparser is configured to perform intra-mode prediction.
 18. The videodecoding apparatus as claimed in claim 1, wherein said reconstructionpipeline is configured to perform parallel decoding on said frames ofvideo data and to reference said line control information to read itemsof macroblock information in parallel from said memory.
 19. The videodecoding apparatus as claimed in claim 18, wherein said reconstructionpipeline is configured to reference said line control information toread said frames of video data in parallel from said memory.
 20. Thevideo decoding apparatus as claimed in claim 18, wherein saidreconstruction pipeline is configured to reference said line controlinformation to read said rows of macroblock information in parallel fromsaid memory.
 21. A method of decoding an encoded video bitstream, saidencoded video bitstream representing frames of video data encoded inrows of macroblocks, the method comprising the steps of: receiving saidencoded video bitstream; interpreting said encoded video bitstream toperform entropy decoding and generate items of macroblock information tobe used for reconstructing said frames of video data; storing said itemsof macroblock information in a memory in the same order in which theyare received in said encoded video bitstream; generating line controlinformation associated with each row of macroblocks, said line controlinformation configured such that sequentially reading said line controlinformation provides access to said items of macroblock information inraster scan order; storing said line control information in said memoryin association with said items of macroblock information; resolvingpredictive encoding between macroblocks of said encoded video streamwhile reading said macroblocks in sequential order; and reconstructingsaid frames of video data with reference to said line controlinformation.
 22. A non-transitory computer readable storage mediumstoring a computer program, which when loaded onto a computing devicecauses said computing device to carry out the method of claim
 21. 23. Avideo decoding apparatus for decoding an encoded video bitstream, saidencoded video bitstream representing frames of video data encoded inrows of macroblocks, said video decoding apparatus comprising: means forreceiving said encoded video bitstream, for interpreting said encodedvideo bitstream to perform entropy decoding, generating items ofmacroblock information to be used for reconstructing said frames ofvideo data, and for storing said items of macroblock information in amemory in the same order in which they are received in said encodedvideo bitstream; means for generating line control informationassociated with each row of macroblocks, said line control informationconfigured such that sequentially reading said line control informationprovides access to said items of macroblock information in raster scanorder, said means for generating line control information for storingsaid line control information in association with said items ofmacroblock information; wherein said means for interpreting isconfigured to resolve predictive encoding between macroblocks of saidencoded video stream while reading said macroblocks in sequential order;and means for reconstructing said frames of video data with reference tosaid line control information.